Add OpenTelemetry middleware by anuraaga · Pull Request #150 · connectrpc/connect-python

anuraaga · 2026-03-03T07:27:56Z

Initial implement with basic tracing / propagation. Follow-ups will add opt-in metadata attributes and metrics.

Some questions

Whether to use a separate package at all. As an "OTel guy" I'm happy to go with native instrumentation built into the library as I did with pyqwest (single dep on opentelemetry-api) but started with middleware since it's more common.
PyPi package name - I went with connectrpc-otel hoping we will publish to connectrpc eventually (not sure the latest on that)
Whether to use a workspace package vs separate repo. It probably won't really depend on much of conenctrpc and could be better to version separately, but it becomes easier to lose track of the repo

Happy with any direction for these.

https://opentelemetry.io/docs/specs/semconv/rpc/connect-rpc/

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>

anuraaga · 2026-03-03T07:28:33Z

connectrpc-otel/connectrpc_otel/_interceptor.py

+            finally:
+                self._finish_span(span, error)
+
+    async def intercept_client_stream(


I only added unit tests for unary for now since I'll probably avoid the stream-type specific interceptors after #149

I'll probably avoid the stream-type specific interceptors after #149

hm, what does #149 have to do with that?

A lot :) Rewrote to metadata interceptor so the logic is constant across stream types now

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>

stefanvanburen · 2026-03-03T15:01:37Z

Whether to use a workspace package vs separate repo. It probably won't really depend on much of conenctrpc and could be better to version separately, but it becomes easier to lose track of the repo

Are we able to keep the upstreamed PyPI version completely separated? I think as long as we can do that, it's probably ok to keep in this repo — I think we avoided it with Go because it's annoying to do multi-module versioning in a single repo, and probably borrowed that approach for the -es repos (like @connectrpc/validate).

If we can do e.g. tag connectrpc-otel/0.1.0 and have that release just that package, probably fine? I don't know exactly how all the recent python tooling works with this sort of set up. Just wanted to get your thoughts before taking a closer look.

anuraaga · 2026-03-04T02:23:26Z

I don't know exactly how all the recent python tooling works with this sort of set up. Just wanted to get your thoughts before taking a closer look.

Yeah that should be quite possible - I don't think there's really any special tooling involved, uv build only builds the current directory's wheel, and we also point the publish github actions to a folder. We should be able to detect the tag name to control this. The only question is if the Releases page becomes confusing with different project cycles there - for example someone subscribed to repo releases would see connectrpc-otel releases. I would suspect those aren't frequent so it may be fine.

To clarify the other repos

I think we avoided it with Go because it's annoying to do multi-module versioning in a single repo

IIRC, the current proposal connectrpc-otel/0.1.0 is actually exactly how to do multi-module for Go. So if it was problematic there, maybe it's something to think about.
and probably borrowed that approach for the -es repos (

I think umbrella projects in Node are super-common, and probably wouldn't even be asking about the approach there ;)

stefanvanburen

I think I'm good with the approach on developing this in-repo, rather than splitting it out elsewhere, so forgive me if some of my comments are off-base; I think we can land this probably as-is and continue to iterate on it until it's ready to release.

I'm mostly coming to this from the context of otelconnect-go, which I already only have a passing familiarity with 😅. It might be nice to further spell out how close we're trying to be to that library, where we're making different decisions, etc.

again, happy to land as-is, just trying to pick your brain so I can wrap my head around this a little better.

stefanvanburen · 2026-03-04T21:20:16Z

connectrpc-otel/connectrpc_otel/_interceptor.py

+RES = TypeVar("RES")
+
+# Workaround bad typing
+_DEFAULT_TEXTMAP_SETTER = cast("Setter[MutableMapping[str, str]]", default_setter)


any upstream issue we ought to follow for this? (I'm not super familiar with the python otel repos.)

Opening up otel-python repos in an IDE, it's all red ;) I think chasing down typing issues there will be a rabbit hole

stefanvanburen · 2026-03-04T21:32:44Z

connectrpc-otel/connectrpc_otel/_semconv.py

+# Vendored in OpenTelemetry semantic conventions for connect-python to avoid
+# unstable imports. We don't copy docstrings since for us they are implementation
+# details and should be obvious enough.


Does otel ever stabilize this stuff (/ have a plan to stabilize this stuff)? Just wondering if we need to leave this interceptor at 0.x until it is (or else consider always populating old values?)

After many many years yeah :) HTTP and DB are finally stable, RPC is at RC. So I think it's safe to use even in a 1.x if we beat them to it

We're definitely not going to use the old attributes for this new library to keep ourselves sane

BTW in terms of import, I wish python had two artifacts, one with stable and one with incubating, but they only have one with _ imports for incubating, and a ton of cruft from before they had the _ concept. I don't expect their current artifact to ever be 1.x.

I filed open-telemetry/opentelemetry-python#4952 to see if this may change

stefanvanburen · 2026-03-04T21:36:27Z

connectrpc-otel/pyproject.toml

@@ -0,0 +1,61 @@
+[project]
+name = "connectrpc-otel"
+version = "0.8.1"


should we land this as 0.1.0?

stefanvanburen · 2026-03-04T21:36:38Z

connectrpc-otel/pyproject.toml

+[project]
+name = "connectrpc-otel"
+version = "0.8.1"
+description = "OpenTelemetry middleware for connectrpc"


s/middleware/interceptor?

Went with instrumentation which is the standard term for this in OTel

stefanvanburen · 2026-03-04T21:38:52Z

connectrpc-otel/connectrpc_otel/_interceptor.py

+            finally:
+                self._finish_span(span, error)
+
+    async def intercept_client_stream(


I'll probably avoid the stream-type specific interceptors after #149

hm, what does #149 have to do with that?

stefanvanburen · 2026-03-04T21:40:27Z

connectrpc-otel/connectrpc_otel/_semconv.py

+
+
+class RpcSystemNameValues(Enum):
+    CONNECTRPC = "connectrpc"


since we support gRPC, would we plan on eventually supporting gRPC via this interceptor? or do that elsewhere?

Yup will add it to this interceptor later

stefanvanburen · 2026-03-04T21:45:26Z

connectrpc-otel/connectrpc_otel/_interceptor.py

+        self,
+        *,
+        propagator: TextMapPropagator | None = None,
+        tracer_provider: TracerProvider | None = None,


would we also support stuff like a MeterProvider and the various knobs from otelconnect-go? (there's a lot of Options and I'm honestly not clear which ones are something we'd intend to support here, versus may be vestigial.)

Yup, will add it with metrics later. Will also consider the options later - some like trust remote, attribute filters, are less common in Python than Go in my experience.

stefanvanburen · 2026-03-04T21:48:52Z

src/connectrpc/_server_sync.py

+            if ra := environ.get("REMOTE_ADDR"):
+                port = environ.get("REMOTE_PORT", "0")


this is WSGI stuff, right?

Actually it's CGI - WSGI was an extension to CGI, so WSGI servers also commonly populate this (except pyvoy since I didn't notice this until writing this PR, will fix it ;) ). Unfortunately WSGI itself doesn't define this at all.

If a server doesn't populate, we don't get it but that's the best we can do - client IP is pretty important in traces to identify bad actors so it's worth the best effort

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>

…nto connectrpc-otel

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>

Add OpenTelemetry middleware

60822c0

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>

anuraaga requested a review from a team March 3, 2026 07:27

anuraaga added 2 commits March 3, 2026 16:29

Sync all packages

90c09ba

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>

Cleanup

912e7ad

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>

anuraaga commented Mar 3, 2026

View reviewed changes

Fix typing

b11dcfd

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>

stefanvanburen approved these changes Mar 4, 2026

View reviewed changes

anuraaga added 4 commits March 5, 2026 10:09

Version

8fc9f42

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>

Merge branch 'main' of https://github.com/connectrpc/connect-python i…

c455314

…nto connectrpc-otel

Use metadata interceptor

2cc8fac

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>

Drift

02527a9

Signed-off-by: Anuraag Agrawal <anuraaga@gmail.com>

stefanvanburen approved these changes Mar 5, 2026

View reviewed changes

anuraaga merged commit 8a1bfc3 into connectrpc:main Mar 6, 2026
34 of 57 checks passed

		if ra := environ.get("REMOTE_ADDR"):
		port = environ.get("REMOTE_PORT", "0")

Conversation

anuraaga commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stefanvanburen commented Mar 3, 2026

Uh oh!

anuraaga commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stefanvanburen left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anuraaga Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

anuraaga commented Mar 3, 2026 •

edited

Loading

anuraaga commented Mar 4, 2026 •

edited

Loading

anuraaga Mar 5, 2026 •

edited

Loading